The approaches described in this section could be pursued but are not necessarily approaches that have previously been conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.
Typically, a data connection between a client and a server is established using a TCP with a three-way handshake. Typically, a three-way handshake procedure is used to synchronize and establish a TCP connection between a client and a server. More specifically, during the three-way handshake procedure, the client initiates the TCP connection by sending a synchronization (SYN) packet to the server. The server acknowledges receipt of the SYN packet by sending a SYN acknowledgement (ACK) packet to the client. Upon receipt of the SYN ACK packet from the server, the client responds with an acknowledgement (ACK) packet, thereby concluding the three-way handshake procedure.
In many cases, the data connection between the client and the server is established through a middle network appliance, such as a load balancer or a security appliance within the server side infrastructure. Server side network appliances may optimize performance and protect the server from unwanted data traffic.
Many existing threat protection systems employed in the network appliances addressing denial of service attacks (DOS and DDOS) use methods based on generating a TCP SYN/ACK cookie to validate a client (also referred to herein as a network device). The TCP SYN/ACK cookie may include a TCP sequence number, which the server sends to the client in a TCP SYN+ACK packet and expects to receive back from the client in the ACK response packet to validate the client and establish a secure communication. Using a traditional TCP SYN/ACK cookie method, the network device can send a synchronization request (SYN), which can be intercepted by the network appliance associated with the server. The network appliance can respond back with a SYN Acknowledgement (SYN ACK), which can include a cookie so that the server is completely unburdened while the connection is being established. A non-attacker network device normally responds back to a SYN ACK, while an attacker network device generates SYN flags but does not proceed to finalize the three-way handshake process.
If the network device responds to the SYN ACK, the network appliance can validate the cookie received back from the network device, generate a new SYN (without cookies), and send it to the server. Therefore, upon successful validation of the network device, there are two TCP connections, one between the network device and the network appliance and another between the network appliance and the server, where the network appliance acts as a proxy and transfers data packets between the two devices.
There is a mode of deployment called direct server return, which is used in the middle network appliances. Using this mode, packets from the network device to the server are transferred through the network appliance, but packets from the server to the network device are not directed through the network appliance in the middle. This is usually used in cases where the traffic from the server to the network device is much higher than the traffic from the network device to the server. Unfortunately, the direct return mode is not possible with traditional TCP SYN/ACK cookie methods because they require constant two-way communications between the network device and the network appliance. Therefore, the network appliance may become a bottleneck for the packets traveling from the server to the network device and introduce more latency than can be tolerated. There is no existing technology to combine the traditional SYN cookie protection with the direct server return mode.